Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Green sirenia or bust! #6976

Merged
merged 28 commits into from
Dec 23, 2024
Merged

Green sirenia or bust! #6976

merged 28 commits into from
Dec 23, 2024

Conversation

dlpierce
Copy link
Contributor

@dlpierce dlpierce commented Nov 22, 2024

Fixes #6894

Summary

Get Sirenia specs passing and add to test matrix

Detailed Description

Many fixes for specs broken in sirenia (fedora 6/valkyrie). See individual commmit messages.

Docker compose files have been reworked to use the same image.

The custom queries added in support of valkyrie statistics support have been converted from SQL to Solr.

@dlpierce dlpierce added notes-bugfix Release Notes: Fixed a bug notes-container Release Notes: Docker, Helm, etc notes-tests Release Notes: Spec or CI changes labels Nov 22, 2024
Copy link

github-actions bot commented Nov 22, 2024

Test Results

    13 files   -     4      13 suites   - 4   2h 49m 36s ⏱️ + 30m 47s
 6 833 tests +   93   6 534 ✅ +   93  299 💤 ± 0  0 ❌ ±0 
18 100 runs  +4 847  17 623 ✅ +4 775  477 💤 +72  0 ❌ ±0 

Results for commit e4231b9. ± Comparison against base commit dd6f355.

This pull request removes 275 and adds 368 tests. Note that renamed tests count towards both.
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplate:0x00007fc97abf8fd8>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplate:0x00007ff3f514ba40>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplateAccess:0x00007fc97a2a8208>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplateAccess:0x00007ff3f5141b80>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to destroy AdminSet: 72b2c66c-35e8-458a-b051-427f6d66913d
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to destroy Hyrax::AdministrativeSet: 07a9c5b3-6dd2-4dad-8561-34fccf70c597
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to edit AdminSet: 584e7bc9-a42f-47b1-aa82-3897e8c0fe0f
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to edit Hyrax::AdministrativeSet: 116f97ac-1652-45dc-8951-5f04cbefd06c
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to update AdminSet: dab78b5e-563f-417d-b2cb-413bc06336cc
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to update Hyrax::AdministrativeSet: 81bd9a07-ccbf-4ad8-85b7-e10db1a3980a
…
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplate:0x00007f0d86240dc0>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplate:0x00007fb1c5aadad8>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplate:0x00007fe8f6881f58>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplateAccess:0x00007f0d852d2d20>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplateAccess:0x00007fb1c5a8c590>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to create #<Hyrax::PermissionTemplateAccess:0x00007fe8ef63f648>
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to destroy AdminSet: 017aa5b5-027f-480e-8a96-c02436dbe9f4
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to destroy Hyrax::AdministrativeSet: 84cdab53-faac-491c-8491-6b90711943cf
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to destroy Hyrax::AdministrativeSet: bab66e0c-8aa3-4a1f-b322-9a02dfd8ee5b
spec.abilities.ability_spec ‑ Hyrax::Ability AdminSets and PermissionTemplates a user without edit access is expected not to be able to edit AdminSet: f88ed61a-13a9-4a5d-9887-a9fab000ddb5
…

♻️ This comment has been updated with latest results.

@dlpierce dlpierce self-assigned this Dec 2, 2024
@dlpierce dlpierce marked this pull request as ready for review December 10, 2024 19:35
@dlpierce dlpierce changed the title WIP Green sirenia or bust! Green sirenia or bust! Dec 10, 2024
@dlpierce dlpierce force-pushed the green-sirenia-or-bust branch from ddf1637 to 2f30836 Compare December 11, 2024 15:14
Ldp::Gone is the error raised by the fedora adapter when the resource is a tombstone after being deleted.
The valkyrie fedora adapter does not convert RDF Date causing comparison failures.
In real use the embargo transaction converts to a datetime, but this factory sets the value directly, so let's make it match real use.
This fixes file_set_indexer_spec which has a bunch of odd setup and mocking, likely from before our factories were equipped to handle building work/file_set/file_metadata stacks. However, it feels right to include here and be treated as an ObjectNotFoundError.
- All 3 test apps use the same container image.
- dassie stays in hyrax-webapp, koppie/sirenia live in hyrax-koppie.
- db_migrate service moved to dev-entrypoint in web service.
- Worker waits to start until rails app is up, should reduce bundle install racing.
- hyrax-engine-dev target now based on hyrax-worker-base.
- Chrome is still a pain and randomly stops responding.
Add worker to depends_on to avoid volume copy race.
Fix Dockerfile style issues
Align chrome service config
Corrected container names for log capture
Now that this spec uses the real valkyrie adapter,
the object coming out of wings is really an anonymous
descendant of the valkyrie model class.
@dlpierce dlpierce force-pushed the green-sirenia-or-bust branch from 2f30836 to b555c49 Compare December 11, 2024 18:48
Copy link
Member

@orangewolf orangewolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great over all, I have a few questions but nothing that needs to stop the show

end_range = end_datetime.blank? ? '*' : end_datetime.utc.xmlschema
args = "system_create_dtsi:[#{start_datetime.utc.xmlschema} TO #{end_range}]"
args += " AND has_model_ssim: (#{models.map { |m| "\"#{m}\"" }.join(' OR ')})" unless models.empty?
Hyrax::SolrService.query(args)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this change the flavor of thing returned? instead of models [Array] is it now SolrHit [Array]?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this should be changed to return a model.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned performance of these statistics related custom queries is going to be terrible on production sized data sets.

bin/worker-entrypoint.sh Outdated Show resolved Hide resolved
@@ -33,82 +32,67 @@ services:
- 1049:1048
volumes:
- ./bin:/app/samvera
- .koppie:/app/samvera/hyrax-webapp
- .koppie:/app/samvera/hyrax-koppie
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can I ask what we accomplish by making this different? its extra mental overhead to remember which is which unless it serves some purpose.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done to better support having both .dassie and .koppie available in the hyrax-engine-dev container image, but in the end it might not be necessary. My thinking was it is easier to know which test app you're in by the directory name, but you make a good point too. I'll look into undoing the hyrax-koppie dir.

@@ -4,6 +4,10 @@
# Run the jasmine tests by running the karma javascript test framework
# The spec will fail if any jasmine tests fails.
RSpec.describe "Jasmine" do
before do
`cd $RAILS_ROOT && bundle exec rake assets:precompile` if ENV['RAILS_ROOT']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is really expensive to do on every spec run... is there anything we can do to mitigate that cost?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find a way to make the JS tests work without compiling. This used to occur during the image build, so it's not a new time sink.

@dlpierce dlpierce force-pushed the green-sirenia-or-bust branch from d3213a3 to 056346f Compare December 20, 2024 15:42
Copy link
Member

@orangewolf orangewolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. Thank you for the revisions. I agree the performance could be an issue on the date queries, but "we aren't doing performance on these" is clearly stated and should be solved with a production load in place.

@orangewolf orangewolf merged commit 4baa51f into main Dec 23, 2024
17 checks passed
@orangewolf orangewolf deleted the green-sirenia-or-bust branch December 23, 2024 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
notes-bugfix Release Notes: Fixed a bug notes-container Release Notes: Docker, Helm, etc notes-tests Release Notes: Spec or CI changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sirenia should run in Github actions and pass
2 participants